<p>
	QuantConnect provides Morningstar fundamentals data for US Equities. Valuation Ratios is daily data. For others like operation ratios and financial statements data are available for multiple periods depending on the property. Please refer to <a href="https://www.quantconnect.com/data#fundamentals/usa/morningstar">Data Library</a> for detailed factors available.
</p>
<p>
	The algorithm is designed to test the significance of one factor each time.
</p>

<div class="section-example-container">

<pre class="python">def Initialize(self):
	self.SetStartDate(2005,01,01)  #Set Start Date
	self.SetEndDate(2012,03,01)    #Set End Date
	self.SetCash(50000)            #Set Strategy Cash
	self.UniverseSettings.Resolution = Resolution.Daily
	self.AddUniverse(self.CoarseSelectionFunction, self.FineSelectionFunction)
	self.AddEquity("SPY") # add benchmark
	self.numOfCourseSymbols = 200
	self.numOfPortfolio = 5
	self._changes = None
	self.flag1 = 1  # variable to control the monthly rebalance of coarse and fine selection function
	self.flag2 = 0  # variable to control the monthly rebalance of OnData function
	self.flag3 = 0  # variable to record the number of rebalancing times
        # store the monthly returns of different portfolios in a dataframe
	self.df_return = pd.DataFrame(index = range(self.numOfPortfolio+1))
        # schedule an event to fire at the first trading day of SPY
	self.Schedule.On(self.DateRules.MonthStart("SPY"), self.TimeRules.AfterMarketOpen("SPY"), Action(self.Rebalancing))
</pre>
</div>
<h3>Step 1: Ranking the stocks by factor values</h3>
<p>
	1. First, we sort the stocks by daily dollar volume and take the top stocks with the highest dollar volumes as our candidates. There is a convenient way using our universe selection API. Universes are refreshed every day by default, but can be refreshed as often as required. This is controlled by the variable <code>UniverseSettings.Resolution</code>. You can refer to the <a href="https://www.quantconnect.com/docs#Universes">documentation</a> for more details.  Here we use <a href="https://www.quantconnect.com/docs#Scheduled-Events">Scheduled events API</a> to trigger code to run at the first trading day each month and use three flag variables to control the rebalancing of <code>CoarseSelection</code>, <code>FineSelection</code> and <code>Ondata</code> functions.
</p>
<p>
	Coarse Universe selection is the built-in universe data provided by QuantConnect which allows you to filter the universe of over 16000 symbols to perform rough filtering before your algorithm. Because coarse selection function takes all the equities including ETFs which have no fundamental data into account, we need the property <code>x.HasFundamentalData</code> to exclude them from our candidate stocks pool.
</p>
<div class="section-example-container">

<pre class="python"># sort the data by daily dollar volume and take the top entries
def CoarseSelectionFunction(self, coarse):
    if self.flag1:
        CoarseWithFundamental = [x for x in coarse if x.HasFundamentalData]
        sortedByVolume = sorted(CoarseWithFundamental, key=lambda x: x.DollarVolume, reverse=True)
        top = sortedByVolume[:self.numOfCourseSymbols]
				return [i.Symbol for i in top]
    else:
        return []
</pre>
</div>

<p>
	2. We extract the factor values of candidate stocks at the beginning of each month and sort the stocks in ascending 
	order of their factor values. Here we use 12-months' total risk-based capital data 
	<code>x.FinancialStatements.TotalRiskBasedCapital.TwelveMonths</code> as an example.
	It is the sum of Tier 1 and Tier 2 Capital. <code>x.Symbol.Value</code> 
	can give the string symbol of selected stock x. Then we save those sorted symbols as <code>self.symbol</code>.
</p>

<div class="section-example-container">

<pre class="python">def FineSelectionFunction(self, fine):
	if self.flag1:
		self.flag1 = 0
		self.flag2 = 1
		# filter the fine by deleting equities wit zero factor value
		filtered_fine = [x for x in fine if x.FinancialStatements.TotalRiskBasedCapital.TwelveMonths != 0 ]
		# sort the fine by reverse order of factor value
		sorted_fine = sorted(filtered_fine, key=lambda x: x.FinancialStatements.TotalRiskBasedCapital.TwelveMonths, reverse=True)
		self.symbol = [str(x.Symbol.Value) for x in sorted_fine]
		# factor_value = [x.ValuationRatios.PERatio for x in sorted_fine]
		self.flag3 = self.flag3 + 1
 	 	return []
	else:
		return []
</pre>
</div>

<h3>Step 2: Compute the monthly return of portfolios</h3>
<p>
	1. At the end of each month, we extract the one-month history close prices of each stock and compute the monthly returns.
</p>

<div class="section-example-container">

<pre class="python">sorted_symbol = self.symbol
self.AddEquity("SPY") # add benchmark
for x in sorted_symbol:
	self.AddEquity(x)
history = self.History(20,Resolution.Daily)
monthly_return =[]
new_symbol_list =[]
for j in range(len(sorted_symbol)):
	try:
		daily_price = []
		for slice in history:
			bar = slice[sorted_symbol[j]]
			daily_price.append(float(bar.Close))
		new_symbol_list.append(sorted_symbol[j])
 		monthly_return.append(daily_price[-1] / daily_price[0] - 1)
	except:
		self.Log("No history data for " + str(sorted_symbol[j]))
		del daily_price
# the length of monthly_return list should be divisible by the number of portfolios
monthly_return = monthly_return[:int(math.floor(len(monthly_return) / self.numOfPortfolio) * self.numOfPortfolio)]
</pre>
</div>


<p>
	2. We divide the stocks into 5 portfolios and compute the average monthly returns of each portfolio. Then we add the monthly return of benchmark "SPY" at the last line of the data frame <code>df_return</code>.
</p>

<div class="section-example-container">

<pre class="python">reshape_return = np.reshape(monthly_return, (self.numOfPortfolio, len(monthly_return)/self.numOfPortfolio))
# calculate the average return of different portfolios
port_avg_return = np.mean(reshape_return,axis=1).tolist()
# add return of "SPY" as the benchmark  to the end of the return list
benchmark_syl = self.AddEquity("SPY").Symbol
history_benchmark = self.History(20,Resolution.Daily)
benchmark_daily_price = [float(slice[benchmark_syl].Close) for slice in history_benchmark]
benchmark_monthly_return = (benchmark_daily_price[-1]/benchmark_daily_price[0]) - 1
port_avg_return.append(benchmark_monthly_return)
self.df_return[str(self.flag3)] = port_avg_return
</pre>
</div>
<h3>Step 3: Generate the metrics to test the factor significance</h3>
<p>
	After getting the monthly returns of portfolios and the benchmark, we compute the average annual return and excess return over benchmark of each portfolio across the whole backtesting period. Then We generate three metrics to judge the significance of each factor.
</p>

<ul>
 	<li>The first metrics is the correlation between the portfolio' returns and their rank. The absolute value of the correlation coefficient should larger than 0.8.</li>
 	<li>If the return of the rank first portfolio larger than the portfolio at the bottom of the return rankings, we define it the win portfolio and the loss portfolio and vice versa. The win probability is the probability that the win portfolio return outperform the benchmark return. The loss probability is the probability that the loss portfolio return underperform the benchmark.  If the factor is significant, both loss and win probability should greater than 0.4.</li>
 	<li>The excess return of win portfolio should be greater than 0.25, while the excess return of loss portfolio should be lower than 0.05.</li>
</ul>

<div class="section-example-container">

<pre class="python">def calculate_criteria(self,df_port_return):
	total_return = (df_port_return + 1).T.cumprod().iloc[-1,:] - 1
	annual_return = (total_return+1)**(1./6)-1
	excess_return = annual_return - np.array(annual_return)[-1]
	correlation = annual_return[0:5].corr(pd.Series([5,4,3,2,1],index = annual_return[0:5].index))
	# higher factor with higher return
	if np.array(total_return)[0] &gt; np.array(total_return)[-2]:
		loss_excess = df_port_return.iloc[-2,:] - df_port_return.iloc[-1,:]
		win_excess = df_port_return.iloc[0,:] - df_port_return.iloc[-1,:]
		loss_prob = loss_excess[loss_excess&lt;0].count()/float(len(loss_excess)) win_prob = win_excess[win_excess&gt;0].count()/float(len(win_excess))
		win_port_excess_return = np.array(excess_return)[0]
		loss_port_excess_return = np.array(excess_return)[-2]
	# higher factor with lower return
	else:
		loss_excess = df_port_return.iloc[0,:] - df_port_return.iloc[-1,:]
		win_excess = df_port_return.iloc[-2,:] - df_port_return.iloc[-1,:]
		loss_prob = loss_excess[loss_excess&lt;0].count()/float(len(loss_excess)) win_prob = win_excess[win_excess&gt;0].count()/float(len(win_excess))
		win_port_excess_return = np.array(excess_return)[-2]
		loss_port_excess_return = np.array(excess_return)[0]
	test_result = {}
	test_result["correelation"]=correlation
	test_result["win probality"]=win_prob
	test_result["loss probality"]=loss_prob
	test_result["win portfolio excess return"]=win_port_excess_return
	test_result["loss portfolio excess return"]=loss_port_excess_return

	return test_result</pre>
</div>

<table class="table qc-table">
<thead>
<tr>
<th style="text-align: center;" colspan="8">Factor Significance Testing Result</th>
</tr>
</thead>
<tbody>
<tr>
<td> Factor</td>
<td> FCFYield</td>
<td> BuyBackYield</td>
<td> PriceChange1M</td>
<td>TrailingDividendYield</td>
<td> EVToEBITDA</td>
<td> RevenueGrowth</td>
<td>BookValuePerShare</td>
</tr>
<tr>
<td> The correlation</td>
<td> -0.936</td>
<td> -0.987</td>
<td> 0.918</td>
<td> -0.981</td>
<td> 0.939</td>
<td>0.89</td>
<td>-0.92</td>
</tr>
<tr>
<td>Win Probability</td>
<td>0.630</td>
<td>0.639</td>
<td> 1</td>
<td> 0.667</td>
<td> 0.722</td>
<td> 0.69</td>
<td>0.69</td>
</tr>
<tr>
<td>Loss probability</td>
<td> 0.426</td>
<td>0.472</td>
<td> 1</td>
<td> 0.518</td>
<td> 0.472</td>
<td> 0.42</td>
<td>0.40</td>
</tr>
<tr>
<td> Excess Return(Win)</td>
<td> 0.324</td>
<td> 0.212</td>
<td>0.303</td>
<td> 0.225</td>
<td> 0.414</td>
<td> 0.23</td>
<td> 0.27</td>
</tr>
<tr>
<td> Excess Return(Loss)</td>
<td> 0.060</td>
<td> 0.037</td>
<td> -1.67</td>
<td> 0.043</td>
<td> 0.042</td>
<td> 0.07</td>
<td> 0.06</td>
</tr>
</tbody>
</table>

<p>
	We choose 4 factors: FCFYield, PriceChange1M, BookValuePerShare and RevenueGrowth.
</p>
